Facial image display apparatus, facial image display method, and facial image display program

ABSTRACT

The present invention provides a facial image display apparatus that can display moving images concentrated on the face when images of people&#39;s faces are displayed. A facial image display apparatus is provided wherein a facial area detecting unit ( 21 ) detects facial areas in which faces are displayed from within a target image for displaying a plurality of faces; a dynamic extraction area creating unit ( 22 ) creates, based on the facial areas detected by the facial area detecting means, a dynamic extraction area of which at least one of position and surface area varies over time in the target image; and a moving image output unit ( 27 ) sequentially extracts images in the dynamic extraction area and outputs the extracted images as a moving image.

TECHNICAL FIELD

The present invention relates to a facial image display apparatus, afacial image display method, and a facial image display program fordisplaying images of people's faces.

BACKGROUND ART

Generally, in cases in which an image display apparatus displays imagedata for a photograph of a person or the like, the entire image isdisplayed. Also, the display of images as a moving image that follows amovement sometimes involves methods such as displaying a degeneratedimage while gradually brightening the entire image, and displaying theentire image by gradually zooming out the image from a specificposition.

DISCLOSURE OF THE INVENTION

Problems the Invention Is Intended to Solve

However, in the conventional example given above, a moving image thatemphasizes the contents of the images cannot be created, and istherefore relatively dull and not entertaining to the user. Anotherproblem is that the user's attention is drawn to areas in the imagehaving relatively little importance.

The present invention was designed in view of these circumstances, andan object thereof is to provide a facial image display apparatus, afacial image display method, and a facial image display program that candisplay moving images concentrated on the face when images of people'sfaces are displayed.

Means for Solving these Problems

The facial image display apparatus according to the present invention,which is intended to resolve these problems, comprises facial areadetecting means for detecting facial areas in which faces are displayedfrom within a target image for displaying a plurality of faces; dynamicextraction area creating means for creating, on the basis of the facialareas detected by the facial area detecting means, a dynamic extractionarea of which at least one of position and surface area varies over timein the target image; and moving image output means for sequentiallyextracting images in the dynamic extraction area and outputting theextracted images as a moving image.

The facial image display method according to the present invention is amethod that a computer is used for executing a step for detecting facialareas in which faces are displayed in a target image for displaying aplurality of faces; a step for creating, on the basis of the facialareas detected by the facial area detecting means, a dynamic extractionarea of which at least one of position and surface area varies over timein the target image; and a step for sequentially extracting images inthe dynamic extraction area and outputting the extracted images as amoving image.

The facial image display program according to the present invention is aprogram that a computer is caused to function as facial area detectingmeans for detecting facial areas in which faces are displayed fromwithin a target image for displaying a plurality of faces; dynamicextraction area creating means for creating, on the basis of the facialareas detected by the facial area detecting means, a dynamic extractionarea of which at least one of position and surface area varies over timein the target image; and moving image output means for sequentiallyextracting images in the dynamic extraction area and outputting theextracted images as a moving image.

An information recording medium according to the present invention is aninformation recording medium that can be read by a computer and thatstores a program that causes a computer to function as facial areadetecting means for detecting facial areas in which faces are displayedfrom within a target image for displaying a plurality of faces; dynamicextraction area creating means for creating, on the basis of the facialareas detected by the facial area detecting means, a dynamic extractionarea of which at least one of position and surface area varies over timein the target image; and moving image output means for sequentiallyextracting images in the dynamic extraction area and outputting theextracted images as a moving image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of the facial imagedisplay apparatus according to an embodiment of the present invention;

FIG. 2 is a function block diagram showing the functions of the facialimage display apparatus according to an embodiment of the presentinvention;

FIG. 3 is a schematic depiction of an example of a target image, whichis processed by the facial image display apparatus according to anembodiment of the present invention;

FIG. 4 is a diagram showing an example of the movement path of a dynamicextraction area in a target image in a case in which the dynamicextraction area determining unit determines a movement path;

FIG. 5 is a diagram showing another example of the movement path of adynamic extraction area in a target image in a case in which the dynamicextraction area determining unit determines a movement path;

FIG. 6 is a graph schematically depicting an example of the movementspeed of a dynamic extraction area in a case in which the dynamicextraction area determining unit determines a dynamic extraction area onthe basis of the movement path;

FIG. 7 is a diagram showing an example of a dynamic extraction areadetermined by the dynamic extraction area determining unit; and

FIG. 8 is a diagram showing an example of an image outputted by themoving image output unit.

BEST MODE FOR CARRYING OUT THE INVENTION

In a facial image display apparatus according to one embodiment of thepresent invention, facial area detecting means detects facial areas inwhich faces are displayed from within a target image that displaysmultiple faces. Dynamic extraction area creating means then creates, onthe basis of the facial areas detected by the facial area detectingmeans, a dynamic extraction area of which at least one of position andsurface area varies over time in the target image. Furthermore, movingimage output means sequentially extracts images in the dynamicextraction area and outputs the extracted images as a moving image.

The facial image display apparatus according to the present embodimentis thereby capable of outputting moving images concentrated on facialareas.

In this arrangement, the dynamic extraction area creating means mayinclude feature area determining means for determining feature areasincluding one or more facial areas, extraction sequence determiningmeans for determining the sequence of extracting the feature areas onthe basis of specific conditions, and dynamic extraction areadetermining means for determining the dynamic extraction area on thebasis of the feature areas and the extraction sequence.

Furthermore, in the case described above, the extraction sequencedetermining means may determine the extraction sequence on the basis ofat least one of the number of faces displayed in the feature areas, thesize of the feature areas, and the position of the feature areas in thetarget image.

The facial image display apparatus according to the present embodimentis thereby capable of outputting moving image that display the featureareas in a sequence corresponding to the characteristics of featureareas including facial areas.

The dynamic extraction area determining means may determine the dynamicextraction area so that position thereof in the target image vary overtime along a movement path that is determined on the basis of thepositions of the feature areas in the target image.

The facial image display apparatus according to the present embodimentis thereby capable of outputting a moving image that sequentiallydisplay parts of the target image while moving in accordance with thefeature areas containing facial areas.

The facial image display apparatus according to the present embodimentmay further include detection time estimating means for estimating thetime needed for the facial area detecting means to detect facial areasin one or multiple other target images in cases in which the movingimage output means outputs the moving images on the basis of a pluralityof target images. The moving image output means may output the movingimage over a time duration determined on the basis of the detection timeestimated by the detection time estimating means, and the facial areadetecting means may detect facial areas contained in said one ormultiple other target images during the period in which the moving imageis outputted by the moving image output means.

The facial image display apparatus according to the present embodimentis thereby capable of outputting moving images that continuously displaythe feature areas displayed in multiple target images, without havingthe user wait depending on the time needed for facial areas to bedetected.

In the facial image display apparatus according to the presentembodiment, the dynamic extraction area creating means may select amoving image pattern from among specific moving image pattern options onthe basis of at least one of the number of detected facial areas, thesize of the detected facial areas, and the position of the facial areasin the target image. The dynamic extraction area creating means may thencreate the dynamic extraction area on the basis of the selected pattern,and the moving image output means may output a moving image based on theselected pattern.

The facial image display apparatus according to the present embodimentis thereby capable of outputting a moving image in a patterncorresponding to the detected facial areas.

Preferred embodiments of the present invention are described hereinbelowwith reference to the diagrams.

The facial image display apparatus according to an embodiment of thepresent invention comprises a control unit 11, a storage unit 12, adisplay unit 13, and an operating unit 14, as shown in FIG. 1.

In this arrangement, the control unit 11 is configured from a CPU, forexample, and is used to execute a program stored in the storage unit 12.The storage unit 12 comprises RAM, ROM, or another such memory elementand/or disk device. The storage unit 12 stores the program that isexecuted by the control unit 11, and data on the target image to beprocessed in the present embodiment. The storage unit 12 also operatesas working memory for the control unit 11.

The display unit 13 is a display or the like, for example, and displaysimages according to instructions outputted from the control unit 11.Upon receiving an instruction operation from the user, the operatingunit 14 outputs the contents of the instruction operation to the controlunit 11. The operating unit 14 may be a keyboard, a mouse, a controllerfor a game console, or the like.

The facial image display apparatus according to the present embodimentis functionally configured including a facial area detecting unit 21, adynamic extraction area creating unit 22, a detection time estimatingunit 26, and a moving image output unit 27, as shown in FIG. 2.

The facial area detecting unit 21 first acquires the target image to beprocessed in the present embodiment. The target image may be stored inthe storage unit 12, or may be acquired by a communication unit (notshown) via a network. The facial area detecting unit 21 then detectsimage areas in which faces are displayed as facial areas from within thetarget image.

Specifically, in cases in which a facial area is detected as arectangular area, for example, the position and size of the facial areaare determined using the sizes of the pixels of the target image asunits, and this information is stored in the storage unit 12. In thiscase, a point in the upper left or in the center of the rectangulararea, for example, is used as a representative point of the facial area,and the position of the facial area is determined as positioncoordinates for the representative point in the target image. The sizeof the facial area is determined as two values, which are the width andheight of the rectangular area. In cases in which a facial area isdetected as a circular area, the center point of the circular area isused as a representative point and the size is determined as one valuerepresenting the radius of the circle. Another option is that the facialarea be determined as an elliptical area. In this case, the center ofthe ellipse is used as a representative point, and the size of thefacial area is specified by values representing the radii of the majorand minor axes.

The method whereby the facial area detecting unit 21 detects facialareas is not particularly limited, and various techniques can be used.For example, the facial area detecting unit 21 may detect facial areasby using statistical data pertaining to a facial shape pattern acquiredin advance through a study. In cases in which facial areas are detectedusing this manner of statistical method, the facial areas are detected,and a likelihood value is obtained pertaining to the possibility thatthe detected areas actually contain components representing a face. Thisvalue is somewhat representative of the likeness of the area to a face,and this facial likeness value may be used later by an extractionsequence determining unit 24. The orientation of the face may bedetermined from the facial shape pattern, and this information may beused later by the extraction sequence determining unit 24.

Furthermore, in addition to detecting facial areas, the facial areadetecting unit 21 may perform a process of identifying the facescontained in the areas. If the user has already registered facialinformation regarding person's face to which previously detected facialarea corresponds, the facial area detecting unit 21 can identify theperson's face in the detected facial area by eigenface method or anothersuch face-identifying technique, for example. This information can beused later when the extraction sequence determining unit 24 determinesthe extraction sequence of the feature areas. In cases in whichphotographic images of the user's family are displayed, for example, itis possible to determine whether or not the display of each person inthe photographs should be carried out according to the person, and alsoto display the images by using a display sequence or display method thatcorresponds to each person.

The facial area detecting unit 21 may also acquire attribute informationpertaining to the attributes of the detected facial areas. Thisattribute information contains information pertaining to the youth (age)of the people displayed in the facial areas and information pertainingto the people's expressions, for example. Specifically, the facial areadetecting unit 21 can acquire this attribute information by inferringthe youth and expressions of the people in the facial areas. This isdone on the basis of the results of a frequency analysis for the facialareas, and on the basis of the positional relationship among featurepoints (positions of the eyes, nose, mouth, and the like) contained inthe facial areas, for example. This attribute information is used laterwhen the extraction sequence determining unit 24 determines theextraction sequence of the feature areas. It is thereby made possible toperform processes for displaying moving images concentrated on facialareas representing children's faces, for example.

The dynamic extraction area creating unit 22 creates dynamic extractionarea on the basis of the facial areas detected by the facial areadetecting unit 21. The dynamic extraction area is an area in the targetimage wherein the position and/or surface area vary over time, and thedynamic extraction area is specified by information involving acorrelation between time and the positions and sizes in the targetimage. The moving image output unit 27 acquires images to be displayedin the outputted moving image by clipping target images on the basis ofthe dynamic extraction area. The facial image display apparatusaccording to the present embodiment is capable of outputting movingimages concentrated on the facial areas by determining the dynamicextraction area on the basis of the facial areas.

The following is a description of an example of a case in which thedynamic extraction area creating unit 22 determines image areascontaining one or more facial areas to be feature areas, and thencreates dynamic extraction area from which feature areas are extractedin a sequence based on specific conditions. In this case, the dynamicextraction area creating unit 22 is functionally configured including afeature area determining unit 23, an extraction sequence determiningunit 24, and a dynamic extraction area determining unit 25.

The feature area determining unit 23 determines feature areas bygrouping facial areas according to specific conditions as necessary.These facial areas are detected by the facial area detecting unit 21.The feature areas are image areas contained in the target image in whichone or more faces are displayed. The feature areas are specified byvalues representing positions and sizes in the target image, similar tothe facial areas.

The following is an example of a method for grouping multiple facialareas. Specifically, in cases in which two facial areas overlap, thegroup is determined to be the same in cases in which the surface areasof the overlapping portions are equal to or greater than a specificthreshold. The group may otherwise be determined to be the same merelyif any areas overlap. If the representative points of two facial areasare denoted by P and Q, the group may be determined to be the same whenthe distance between P and Q is equal to or less than a specificthreshold. Two facial areas may also be determined to be in the samegroup in cases in which either the distance between the adjacent sidesis equal to or less than a specific threshold, or the distance is equalto or less than a value determined from a specific ratio relating to thesizes of the facial areas. The adjacent sides herein are a set of twosides in which the distance between the side of one facial area and theside of another facial area is smallest, and which are selected fromsides constituting the external peripheries of the two facial areas(sides of a rectangle that circumscribes the facial areas in cases inwhich the facial areas are circular or elliptical). The feature areadetermining unit 23 determines the feature areas on the basis of one ormore facial areas that are determined to belong to the same groupaccording to these conditions. The feature areas may be of a size thatincludes all of the target facial areas, or a size that includes atleast a specific percentage of the facial areas. The feature areas mayalso be of a size that includes areas equivalent to a specific number ofpixels surrounding the facial areas. Otherwise, the size of the featureareas may be determined according to the orientation of the facescontained in the facial areas so that areas below the faces, forexample, are included proportionate to a number of pixels determinedaccording to a specific ratio relating to the size of the facial areas.It is thereby possible to determine feature areas including people'sfaces from the chest up, for example. In cases of an image area in whicha feature area contains only one facial area, the shape and size of thefeature area may coincide with the facial area.

In cases in which the feature area determining unit 23 determines afeature area containing a single facial area, the size of the featurearea may be determined according to the size of the facial areacontained in the feature area. For example, in cases in which thefeature area determining unit 23 uses, as a feature area, an areacontaining areas proportionate to a specific number of pixelssurrounding the facial area as previously described, this specificnumber of pixels is determined based on the size of the facial area. Asa specific example, the feature area determining unit 23 determines thespecific number of pixels to be a smaller value as the facial areasbecome larger, and a greater value with as the facial areas becomesmaller. The sizes of the feature areas determined by the feature areadetermining unit 23 will then be relatively similar and will be devoidof nonuniformities comparable to the sizes of the facial areas detectedby the facial area detecting unit 21. It is thereby possible to preventthe size of the dynamic extraction area from rapidly changing over timein cases in which the dynamic extraction area determining unit 25determines dynamic extraction area that correspond to a moving imagepattern that is zoomed in on the feature areas, for example, as isdescribed later.

The feature area determining unit 23 determines feature areas so thatthe parts that represent faces contained in the target image areincluded in either of the feature areas. For example, in the targetimage I1 depicted schematically in FIG. 3, a feature area A containingthree facial areas, a feature area B containing one facial area, and afeature area C containing two facial areas are obtained. The dynamicextraction area creating unit 22 then creates dynamic extraction area sothat the feature areas are extracted based on a specific sequence inaccordance with the following process.

The extraction sequence determining unit 24 determines the feature areasto be extracted by the moving image output unit 27 from among thefeature areas determined by the feature area determining unit 23, sortsthe feature areas to be extracted on the basis of specific conditions,and determines the extraction sequence.

Specifically, the extraction sequence determining unit 24 calculatesscore values for each of the feature areas, for example, on the basis ofvalues representing the characteristics of the feature areas. Thesescore values are then used as a basis to determine whether or not thefeature areas are to be extracted, and to determine the sequence withwhich the feature areas will be extracted. The score values can becalculated based on the number of faces displayed in the feature areas,the surface areas of the feature areas, or other such factors. Also, thescore values may be determined using values pertaining to the faciallikenesses, information pertaining to whose face is displayed, and othersuch information obtained by the facial area detecting unit 21 for thefacial areas included in the feature areas.

The extraction sequence determining unit 24 may also determine the scorevalues on the basis of the attribute information acquired by the facialarea detecting unit 21 for the facial areas contained in the featureareas.

Furthermore, the score values may be calculated using a combination ofinformation pertaining to the feature areas. For example, in cases inwhich the score values are calculated based on the number of facesdisplayed in the feature areas and the surface areas of the featureareas, the score values can be calculated from the following formulas,wherein the score values of the feature areas A, B, C in the targetimage I1 are SC_(A), SC_(B), SC_(C).

SC _(A)=(S _(A) ×a1)+(N _(A) ×a2)

SC _(B)=(S _(B) ×a1)+(N _(B) ×a2)

SC _(C)=(S _(C) ×a1)+(N _(C) ×a2)

The symbols N_(A), N_(B), N_(C) are the numbers of faces displayed inthe feature areas A, B, C, and the symbols S_(A), S_(B), S_(C) arevalues representing the surface areas of the feature areas A, B, C. Thesymbols a1, a2 are weighting factors determined in advance.

After the score values for the feature areas are calculated, theextraction sequence determining unit 24 determines the extractionsequence. Specifically, the feature areas are sorted according to thescore value, for example, and are extracted according to the sortedsequence. The feature areas may also be excluded from being extractedfor score values equal to or less than a specific threshold. Forexample, in the example given above, if the relationshipsSC_(A)>SC_(B)>SC_(C), SC_(B)>SC_(min), and SC_(min)≧SC_(C) hold for aspecific threshold SC_(min), the extraction sequence determining unit 24excludes the feature area C from the target to be processed by thedynamic extraction area determining unit 25, and determines anextraction sequence so that the feature areas A, B are extracted withthis sequence. In the case of images of scenic photographs, for example,it is thereby possible to exclude small faces in the scenery from thetargets to be processed.

Instead of determining the extraction sequence according to the scorevalues, the extraction sequence determining unit 24 may determine theextraction sequence according to information pertaining to theorientation of the faces or the positions of the feature areas. In thiscase, the extraction sequence may be determined so that extractionbegins with the feature areas on the left side of the target image, forexample. Also, in some cases it is possible to acquire informationpertaining to the distance from the camera to the photographed subjectin the target image on the basis of information pertaining to a focus oron the basis of overlapping of the photographed images. In such cases, asequence of aligning the facial areas from the front to the rear, orfrom the rear to the front, may be determined based on this information,and the extraction sequence may be determined according to thissequence. The extraction sequence may also be determined randomlyregardless of this type of information on the feature areas.Furthermore, score values may be calculated for the feature areas in thesame manner as the previously described example, and feature areas inwhich the score values are equal to or less than a specific thresholdvalue may be removed from the target to be extracted by the moving imageoutput unit 27, even when the extraction sequence is determinedregardless of the score values.

The following is a description of a specific example of a method fordetermining the extraction sequence on the basis of the positions of thefeature areas. For example, the extraction sequence determining unit 24excludes feature areas contained in a specific range of the target imagefrom the targets of extraction. In this case, feature areas near theexternal periphery of the target image, for example, can be ignored inthe creation of the dynamic extraction area. The extraction sequencedetermining unit 24 may detect the two feature areas farthest from eachother from among the feature areas that have not been excluded from thetarget of extraction, and the extraction sequence determining unit 24may then determine the extraction sequence so that one of these twofeature areas is extracted first, and the other is extracted last.

In addition to determining the feature areas to be extracted, theextraction sequence determining unit 24 may reset the feature areas sothat the facial areas contained in two or more feature areas to beextracted are contained in one feature area in cases in which thefeature areas to be extracted fulfill specific conditions. When thenumber of feature areas to be extracted is equal to or less than aspecific number, for example, the position and size of a feature area isset instead so that all of the facial areas are contained in the featurearea. Alternatively, in cases in which the distance between therepresentative points of two certain feature areas is equal to or lessthan a specific threshold, a feature area may be set instead so as toinclude all of the facial areas contained in these two feature areas. Itis thereby possible to extract more preferable feature areas inaccordance, for example, with cases in which there are few feature areasto be extracted, or the like.

The dynamic extraction area determining unit 25 determines dynamicextraction area in the target image of which the position and/or surfacearea vary over time. The determination is made on the basis of thefeature areas determined by the feature area determining unit 23 and theextraction sequence determined by the extraction sequence determiningunit 24. Specifically, the dynamic extraction area is determined so thatthe feature areas are extracted according to the extraction sequence.The dynamic extraction area determining unit 25 determines the dynamicextraction area on the basis of the pattern of the moving imageoutputted by the moving image output unit 27. The shape of the dynamicextraction area is determined according to the pattern of the movingimage.

The dynamic extraction area determining unit 25 determines the positionand surface area of the dynamic extraction area as follows in cases inwhich the pattern of the moving image outputted by the moving imageoutput unit 27 is based on the below-described morphing process or afade-in/fade-out process. Specifically, the position and surface area inthe target image are determined so as to include a feature areacorresponding to the first extraction sequence. And, the dynamicextraction area is determined so as to change the position and surfacearea in the target image and include a feature area corresponding to thenext extraction sequence after a time duration determined by specificconditions has passed. These steps are repeated to sequentially changethe dynamic extraction area to position and surface area containingfeature areas in accordance with the extraction sequence determined bythe extraction sequence determining unit 24.

The above-described time duration may be a predetermined time, or may bedetermined based on a reference output time T_(min) in which the entiremoving image created based on the target image is outputted. Thereference output time T_(min) may be a predetermined time, or a timedetermined based on the number of displayed feature areas. The timeduration may also be determined based on the reference output timeT_(min) and an estimated time calculated by the below-describeddetection time estimating unit 26.

Another example of a pattern for the moving image outputted by themoving image output unit 27 is one in which a moving image that movesthrough the target image is outputted by moving the dynamic extractionarea along a movement path determined based on the positions of thefeature areas in the target image. Hereinbelow a detailed description isgiven of the process whereby the dynamic extraction area determiningunit 25 determines the dynamic extraction area in this case.

First, the dynamic extraction area determining unit 25 determines themovement path for moving the dynamic extraction area in the targetimage. The movement path is determined so as to pass through therepresentative points of the feature areas in the sequence determined bythe extraction sequence determining unit 24, for example. The movementpath may be a broken line that connects the representative points in astraight line, or a spline curve that passes through the representativepoints. The movement path may also be determined using a Bezier curve,but in this case, the Bezier curve must be corrected at the end so thatthe dynamic extraction area passes through the feature areas. The movingimage output unit 27 can thereby output a moving image that displays thefeature areas in a specific sequence, because the dynamic extractionarea determining unit 25 determines the dynamic extraction area so as topass through the feature areas to be extracted according to a specificsequence.

FIG. 4 is a diagram showing an example of the movement path when thefeature areas A, B, C are displayed in this order in the example of thetarget image I1 shown in FIG. 3. In this diagram, the representativepoints of the feature areas A, B, C are C_(A), C_(B), C_(C),respectively, and the movement path constitutes a curved line R thatjoins these three points. The dynamic extraction area is determined soas to move along the movement path R through the points in the orderC_(A), C_(B), C_(C).

The movement path does not necessarily need to pass through therepresentative points of the feature areas. For example, the movementpath may be determined so that the distance h to the movement path isequal to or less than a specific threshold for all points on theexternal peripheries of the feature areas. The size of the dynamicextraction area is then determined based on the maximum distance h_(max)of these distances h. The moving image output unit 27 is thereby capableof outputting a moving image displaying the feature areas bysequentially acquiring images clipped from the dynamic extraction area.This can be achieved even when the movement path does not pass throughthe representative points of the feature areas.

Specifically, the positions of the representative points is approximatedby a straight line, for example, the straight line is translated to makethe maximum distance h_(max) as small as possible, and the resultingstraight line is used as the movement path. It is thereby possible tooutput a moving image displaying the feature areas by moving the dynamicextraction area along this straight line in cases in which the featureareas are aligned along a substantially straight line, and theextraction sequence determining unit 24 has determined the extractionsequence of the feature areas on the basis of the positions of thefeature areas. FIG. 5 is a diagram showing an example of a rectilinearmovement path R′ in the case of the target image I1 shown in FIG. 3,wherein the feature area C is not to be extracted and the feature areasA, B are to be displayed in this order. The movement path R′ isdetermined so as to pass through the middle between the lower border ofthe feature area A and the upper border of the feature area B, and themaximum distance h_(max) corresponds to the distance between themovement path R′ and the upper border of the feature area B, and also tothe distance between the movement path R′ and the lower border of thefeature area A. In this case, the dynamic extraction area is determinedas rectangular area whose longitudinal dimension is equal to two timesh_(max).

When the movement path is determined, the dynamic extraction areadetermining unit 25 determines the dynamic extraction area so that theposition of the dynamic extraction area is sequentially varied along themovement path. Specifically, when the representative point of thedynamic extraction area is set to the area center point in the samemanner as the feature areas, the position of the dynamic extraction areais sequentially varied so that the representative point move through theaforementioned movement path. The initial position of the dynamicextraction area may be the same as the position of the first featurearea to be displayed, or the position may be the point where themovement path intersects the external periphery of the target image.When the initial position is the point where the movement pathintersects the external periphery of the target image, the moving imageoutput unit 27 begins extracting the image starting with the peripheryof the target image.

In cases in which the dynamic extraction area determining unit 25sequentially varies the position of the dynamic extraction area, thespeed with which the dynamic extraction area moves through the movementpath is represented by the length of the path through which the positionof the dynamic extraction area moves from the extraction of one image tothe next. This dynamic extraction area speed may be set in advance ormay be calculated by a specific method. Specifically, the speed iscalculated by dividing the length of the movement path from one featurearea to the next by the number of frame images outputted by the movingimage output unit 27 during a predetermined output time T, for example.In this case, the dynamic extraction area moves at a constant speedthrough the movement path from one feature area to the next.

The speed through the movement path may also be calculated so as tolengthen the time during which areas in the vicinity of the featureareas are displayed. Specifically, the speed through the movement pathis determined according to the schematic graph shown in FIG. 6 in casesin which the dynamic extraction area is moved from the feature area A tothe feature area B, for example. The horizontal axis of the graphrepresents the time taken to move from the feature area A to the featurearea B, wherein time 0 is the point at which the dynamic extraction areais positioned at the representative point of the feature area A, andtime T is the point at which the dynamic extraction area is positionedat the representative point of the feature area B. The vertical axisrepresents the distance through the movement path from the feature areaA to the feature area B, wherein the movement distance 0 is the positionof the feature area A, and the movement distance L is the position ofthe feature area B. In the example of the movement path shown in FIG. 4,the length of the curved line R between point C_(A) and point C_(B)corresponds to L. The graph in FIG. 6 has a gentle slope near themovement distances 0 and L, and a steep slope near the middle.Therefore, when the dynamic extraction area is determined based on themovement distance determined according to this graph, the dynamicextraction area moves through the vicinities of the feature areas at alower speed, and moves through positions distant from the feature areasat a higher speed. It is thereby possible to output a moving image that,in particular, moves slowly through the vicinities of faces while movingthrough the target image, and to obtain a display that concentrates onfaces to a greater degree. The speed of the dynamic extraction area maybe determined according to the graph when the first and last featureareas are extracted, and the speed of the dynamic extraction area may bedetermined so that the area moves at a constant speed when the featureareas in the middle are extracted.

The output time T, which represents the time taken by the dynamicextraction area to move from one feature area to the next, may be aspecific time or may be determined based on the value of the movementdistance L. The output time T may also be determined based on theestimated time calculated by the detection time estimating unit 26,described later.

In addition to sequentially changing the position of the dynamicextraction area, the dynamic extraction area determining unit 25 mayalso sequentially change the size of the dynamic extraction area so thatthe dynamic extraction area is large enough to include the featureareas. For example, a feature area is assumed to be rectangular, and thesize of the feature area A is represented by a width w_(A) and a heighth_(A), while the size of the feature area B is represented by a widthw_(B) and a height h_(B). In this case, a dynamic extraction area ismoved from the feature area A to the feature area B while, at the sametime, the width of the dynamic extraction area gradually changes fromw_(A) to w_(B) and the height of the dynamic extraction area graduallychanges from h_(A) to h_(B). It is possible to output a moving image inwhich the display range of the target image changes in accordance withthe sizes of the feature areas. This is achieved by clipping the targetimage at dynamic extraction area that has changed in size. The amount ofchange in the widths and heights in this case may be determined to bealways constant or to vary in accordance with the movement speed of thedynamic extraction area.

In some cases, the facial image display apparatus according to thepresent embodiment detects facial areas from multiple target images, andcontinuously outputs moving images created based on the facial areas. Insuch cases, the detection time estimating unit 26 estimates the timeneeded to detect the facial areas contained in one or more targetimages. The following is a description of examples of continuous outputsof moving images created based on two target images I1, I2,respectively. A moving image A1 is an image obtained by sequentiallyextracting images in the dynamic extraction area created based on thetarget image I1, and a moving image A2 is an image obtained bysequentially extracting images in the dynamic extraction area createdbased on the target image I2. The detection time estimating unit 26first estimates the time needed to detect facial areas in the targetimage I2. The resulting estimated time T_(next) is then used to adjustthe output time when the dynamic extraction area creating unit 22determines the output time of the moving image A1, which is outputted bythe moving image output unit 27 immediately before the moving image A2.

Specifically, the detection time estimating unit 26 calculates anestimated time on the basis of the file size, resolution, or anotherattribute of the target image, for example. The estimated time may alsobe calculated according to a coarse recognition process that easilyrecognizes faces in the target image beforehand at a low resolution.Furthermore, the detection time estimating unit 26 may estimate the timeof processes needed to begin outputting the moving image A2 for thetarget image I2 other than the process for the facial area detection bythe facial area detecting unit 21, such as the process for the facialareas to be grouped to determine the feature areas, and the process todetermine the sequence with which the determined feature areas areextracted. These times may be included in the calculated estimated timeT_(next).

The time estimating unit 26 is not limited to being used for the targetimage outputted subsequent to the moving image currently being outputtedby the moving image output unit 27 as the target for calculating theestimated time. For example, estimated times for multiple target imagesare calculated in cases in which the processes for detecting facialareas in two or more target images are performed in parallel. Thus,estimated times are calculated for processes pertaining to one or moretarget images yet to be displayed.

The following is a description of controlling the output time of amoving image in a case in which the moving image output unit 27continuously outputs moving images A1, A2 that display facial areasdetected from the two target images I1, I2. When the reference outputtime T_(min) for outputting a moving image A1 based on the target imageI1 falls below the estimated time T_(next) needed to detect facial areasfor the next moving image A2, the dynamic extraction area creating unit22 uses T_(next) as the output time for the moving image A1. The movingimage output unit 27 executes a process for outputting the moving imageA1 while, at the same time, the facial area detecting unit 21 performs aprocess for detecting facial areas in the subsequently displayed targetimage I2. It is thereby possible to make the preparations necessary tooutput the moving image A2 before the output of the moving image A1 iscomplete, and moving images can be continuously displayed without havingthe user wait.

The detection time estimating unit 26 may update the estimation of theremaining time needed to detect facial areas in the target image I2 atone or more specific timings. The estimation may be updated even whilethe moving image output unit 27 is still in the process of outputtingthe moving image A1 on the basis of the estimated time for the nexttarget image that has already been calculated. This re-calculatedestimated time is used any time to adjust the remaining output time forthe moving image A1 being outputted by the moving image output unit 27.In this case, the remaining output time for the moving image is adjustedas a result of the dynamic extraction area creating unit 22 recreatingthe dynamic extraction area on the basis of the re-calculated estimatedtime.

Specifically, in cases in which the remaining output time for the movingimage A1 is T_(now) at a certain point in time so that T_(rest)>T_(now),for example, the dynamic extraction area determining unit 25 makesadjustments so that the remaining output time for the moving image A1 isgreater than T_(rest) by revising the dynamic extraction area so thatthe time at each position is increased when the dynamic extraction areachanges position. The wait time for the user to view the moving imagescan thereby be eliminated by appropriately adjusting the output time ofthe moving image A1 even in cases in which the process for the movingimage A2 requires more than the initial estimated time.

Furthermore, in some cases, facial area detection and other processesfor the next target image I2 require more than the estimated timecalculated by the detection time estimating unit 26. As a result, outputof the next moving image A2 cannot begin at the point when output of themoving image A1 is completed. In such cases, the moving image outputunit 27 may either wait in its current state until the necessaryprocesses are completed, or may repeatedly output the moving image A1.Another option is to output a moving image having another pattern inwhich the facial areas contained in the target image I1 are displayed.

In some cases, the pattern for a moving image in which dynamicextraction area is moved along a movement path requires more time thanthe estimated time initially calculated for the facial areas to bedetected in the next target image. In such cases, adjusting the movementspeed of the dynamic extraction area can increase the output time of thecurrent moving image. Another option is to increase the output time ofthe current moving image A1 by changing the movement path in the targetimage I1 so as to increase the distance over which the dynamicextraction area moves.

The moving image output unit 27 sequentially extracts images in thedynamic extraction area by clipping the target image on the basis of thedynamic extraction area created by the dynamic extraction area creatingunit 22. The extracted images are then processed as necessary based on aspecific moving image pattern to create a moving image, and this movingimage is outputted to the display unit 13.

The following is an example of a pattern of a moving image outputted bythe moving image output unit 27. Specifically, first, an image isextracted from a dynamic extraction area containing the feature areathat is first in the extraction sequence, and then an image is createdin which this first image is gradually changed to an image containingthe second feature area by a morphing process. Images created in thismanner are sequentially displayed, resulting in a moving image in whicheach image gradually changes to an image in the dynamic extraction areacontaining the next feature area. A moving image that sequentiallydisplays the feature areas can be outputted by repeating this sequencewhile changing the position and surface area of the dynamic extractionarea to the positions and surface areas containing feature areas.

The moving image pattern may also cause the images containing featureareas to fade in and out. In this case, first, an image is extractedfrom a dynamic extraction area containing the feature area that is firstin the extraction sequence, this image is faded out, and then an imageis sequentially outputted in which the second feature area in theextraction sequence is faded in. A moving image that sequentiallydisplays feature areas can be outputted by repeating this sequence whilechanging the position and surface area of the dynamic extraction area tothe positions and surface areas containing feature areas, similar to thepattern for the morphing process. In this case, the fade-in/fade-outeffect is achieved by gradually changing the brightness of the extractedimages, for example. The moving image may also be designed so that theextracted feature areas are made transparent by gradually increasing thetransparency of the image, and the transparency of the next feature areais gradually reduced from a completely transparent state.

In the pattern for moving dynamic extraction area along the movementpath, a moving image can be outputted that moves through target imagesin which images extracted from the dynamic extraction area are eitheroutputted directly without being processed, or are outputted in anenlarged or compressed format.

The size of the dynamic extraction area normally changes over time onthe basis of the size of the feature areas, but in cases in which themoving image output unit 27 sequentially outputs images extracted fromthe dynamic extraction area, another option is for images acquired byclipping to be enlarged or compressed in accordance with the size ofimages that can be displayed by the display unit 13, whereby images ofthe same size can be displayed on the display unit 13.

The pattern used by the moving image output unit 27 may be eitherselected by the user or selected randomly from the multiple moving imagepatterns described above. Another possibility is for the pattern to bedetermined based on the number of displayed feature areas. For example,a fade-in/fade-out pattern may be used to display the moving image whenthe number of displayed feature areas is equal to or greater than aspecific number, and the moving image may be displayed by moving thedynamic extraction area when the number of displayed feature areas isless than the specific number.

A moving image based on a specific pattern so that the entire targetimage is displayed may be outputted in cases in which the target imageeither does not contain any feature areas to be extracted or containsonly one feature area to be extracted. Furthermore, a target image thatdoes not contain feature areas may be excluded from the moving imagesoutputted by the moving image output unit 27.

According to the embodiment of the present invention described above, amoving image created by concentrating on facial areas can be displayedwhen images representing faces are displayed.

In the present embodiment, the feature area determining unit 23 todetermine feature areas by grouping facial areas, but the feature areadetermining unit 23 need not be included. In such cases, a moving imageconcentrated on faces can be displayed by executing the processes of thepresent embodiment while using the facial areas detected by the facialarea detecting unit 21 as feature areas each representing one face.

In addition to the moving image patterns described above, the facialimage display apparatus according to the present embodiment is alsocapable of outputting moving images having the following pattern.Specifically, a zoom-in/zoom-out pattern may be used for one or morefeature areas containing facial areas. In this case, the dynamicextraction area determining unit 25 first determines the position andsurface area of the dynamic extraction area so as to include the entiretarget image. The dynamic extraction area is then determined so that theposition and surface area of the dynamic extraction area is graduallychanged to the smallest size capable of containing the first featurearea to be extracted. For example, in the target image I1 displayedschematically in FIG. 7, when the user wishes to output a moving imagethat zooms in on the feature area A, the dynamic extraction area isgradually changed from an area encompassing the entire target image toan area encompassing only the feature area A. The arrows displayed withdashed lines in the diagram indicate the extent to which the dynamicextraction area is changed in the target image. The position and surfacearea of the dynamic extraction area are then gradually changed in theopposite manner to a size including the entire target image.Furthermore, in the case in which multiple feature areas are to beextracted, the dynamic extraction area is determined so as to repeat thedescribed change in proportion to the number of feature areas. Themoving image output unit 27 enlarges the images in the dynamicextraction area determined by the dynamic extraction area determiningunit 25 to a specific size regardless of the size of the dynamicextraction area, and outputs the enlarged images. It is thereby possibleto output a moving image that displays feature areas according to azoom-in/zoom-out pattern.

The facial image display apparatus according to the present embodimentmay also be designed to output a moving image for each target image witheither a zoom in or a zoom out pattern alone.

Another example of a moving image pattern is a wipe-in/wipe-out patternthat may be used for one or more feature areas. In this case, thedynamic extraction area is determined in the same manner as in thezoom-in/zoom-out pattern. The moving image output unit 27 createsoutputted images in the following manner. Specifically, the outputtedimage is of the same size as the entire target image I1, the image inthe dynamic extraction area is displayed in the outputted image withoutchanging the position and surface area of the dynamic extraction area,and the other area is filled out with black or another specific color.These images are sequentially created and outputted along with thechange in the dynamic extraction area. For example, when the displaywipes in on the feature area A in the target image I1, an image isoutputted such as is shown schematically in FIG. 8. In this diagram, thearrows displayed with dashed lines represent the process of change inthe dynamic extraction area. The feature area and dynamic extractionarea have an elliptical shape in the illustrated example. It is therebypossible to output a moving image that displays feature areas with awipe-in/wipe-out pattern.

The facial image display apparatus according to the present embodimentmay be designed to output a moving image for each target image witheither a wipe-in or a wipe-out pattern alone when.

Another example of a moving image pattern that may be used to outputmoving images is a moving image pattern that uses the Lens Blur effect.For example, in this moving image pattern, the dynamic extraction areais determined in the same manner as in the wipe-in/wipe-out patterndescribed above. The moving image output unit 27 creates outputtedimages as follows. Specifically, the outputted image is of the same sizeas the entire target image I1, the image in the dynamic extraction areais displayed in the outputted image without changing the position andsurface area of the dynamic extraction area, and the image in the otherarea is the image corresponds to the target image I1 with specific imageprocessing. This specific image processing creates a Lens Blur effect inthe other area. Specifically, with respect to each pixel in the area,the average value of the pixel values for that pixel and its surroundingpixels is calculated, and the pixel values are replaced with thecalculated averages. The image in area other than the dynamic extractionarea thereby goes out of focus and takes on a blurry appearance.

The moving image output unit 27 may gradually vary the specificparameters used in the image processing for creating the Lens Blureffect. For example, the moving image output unit 27 repeats the processof creating outputted images while increasing the range of thesurrounding pixels whose average pixel values are calculated. The movingimage output unit 27 is thereby capable of outputting moving images inwhich areas other than the feature areas appear to go out of focus. Thistype of moving image pattern allows for a display in which areas that donot display faces go out of focus, whereby areas displaying faces standout. In this moving image pattern using a Lens Blur effect, the dynamicextraction area may be a fixed area that corresponds to a singleselected feature area and does not change in position or size.

The following is a description of another example of the process wherebythe facial image display apparatus according to the present embodimentselects a moving image pattern from the moving image patterns previouslydescribed.

For example, the dynamic extraction area creating unit 22 selects amoving image pattern from specific moving image pattern options on thebasis of at least one of the number of facial areas detected by thefacial area detecting unit 21, the sizes of the facial areas, and thepositions of the facial areas within the target image. The dynamicextraction area creating unit 22 then creates dynamic extraction area onthe basis of the selected pattern. The moving image output unit 27subjects the dynamic extraction area to clipping and image processing onthe basis of the moving image pattern selected by the dynamic extractionarea creating unit 22, and a moving image is outputted.

In this case, specific examples of the pattern options that can beselected by the dynamic extraction area creating unit 22 include thepreviously described moving image pattern (panning image pattern) inwhich the dynamic extraction area is moved along a movement path, amoving image pattern that uses a morphing process, a fade-in/fade-outmoving image pattern, a zoom-in/zoom-out moving image pattern, awipe-in/wipe-out moving image pattern, a moving image pattern that usesa Lens Blur effect, and the like. One of the pattern options may also bea combination of these moving image patterns. For example, the patternoption may be a moving image pattern that zooms in on the feature areawhile a Lens Blur effect is applied to the areas outside of the featurearea.

Specifically, the dynamic extraction area creating unit 22 selects amoving image pattern from options that include a panning image patternwhen, for example, multiple facial areas are detected, and the dynamicextraction area creating unit 22 selects a moving image pattern fromoptions that do not include a panning image pattern when only one facialarea is detected.

A moving image pattern may also be selected from these options even whenno facial areas are detected. In this case, the dynamic extraction areacreating unit 22 may create dynamic extraction area not on the basis offeature areas, but rather, for example, on the basis of the centerposition of the target image, or on the basis of a reference positionselected randomly from specific position options.

The dynamic extraction area creating unit 22 may randomly select amoving image pattern from the selectable pattern options. The movingimage pattern may also be selected according to the moving image patternused in the moving image outputted by the moving image output unit 27prior to the upcoming moving image. For example, the dynamic extractionarea creating unit 22 may select a different moving image pattern thanthe one used in the moving image outputted immediately prior to theupcoming moving image, and may use this selected moving image patternfor the upcoming moving image. It is thereby possible to avoid acontinuous moving image based on the same pattern as when moving imagesbased on the target images are continuously outputted, and monotonousdisplays can be prevented.

The facial image display apparatus according to the present embodimentmay perform the following process when displaying moving images based onmultiple target images. Specifically, the moving image output unit 27outputs a moving image in which an image outputted during a specifictime at the end of the currently displayed moving image is graduallyreduced in brightness, and the moving image fades out. The image duringa specific time at the beginning of the subsequently displayed movingimage is then gradually increased in brightness, and the moving imagefades in.

Alternatively, the moving image output unit 27 may combinesemi-transparent displays of the outputted image during a specific timeat the end of the currently displayed moving image, and the outputtedimage during a specific time at the beginning of the subsequentlydisplayed moving image. In this case, the two outputted images may becombined while the transparency of the outputted image in the currentlydisplayed moving image is gradually increased, and the transparency ofthe outputted image in the subsequently displayed moving image isgradually reduced. A moving image can thereby be displayed wherein thecurrently displayed moving image fades out while, at the same time, thesubsequently displayed moving image fades in.

1. A facial image display apparatus comprising: a detector that detectsareas containing faces in a plurality of target images each displayingone or more faces, wherein each target image is a still image, whereinthe detector identifies one or more persons corresponding to the areascontaining faces and creates, using the areas containing faces, a paththrough the plurality of target images connecting the detected areascontaining faces according to the one or more identified persons; and amoving image creating unit that creates a moving image in which a cameramoves through the plurality of target images according to the path,wherein the moving image contains the areas containing faces from theplurality of target images.
 2. The facial image display apparatusaccording to claim 1, wherein the detector further acquires attributeinformation pertaining to the areas containing faces.
 3. The facialimage display apparatus according to claim 2, wherein the attributeinformation comprises information pertaining to an age or an expressioncorresponding to each of the areas containing faces.
 4. The facial imagedisplay apparatus according to claim 2, wherein the detector creates thepath based on the acquired attribute information pertaining to the areascontaining faces.
 5. The facial image display apparatus according toclaim 4, wherein the detector creates the path according to an age ofthe one or more identified persons.
 6. The facial image displayapparatus according to claim 1, wherein the moving image creating unitselects a pattern from among specific moving image pattern options onthe basis of at least one of the number of detected facial areas.
 7. Thefacial image display apparatus according to claim 1, wherein the movingimage zooms in on the detected facial areas.
 8. The facial image displayapparatus according to claim 1, further comprising: a time estimatorthat estimates a time needed to detect facial areas in each of theplurality of target images, wherein the facial image display apparatusoutputs the moving image on a display on the basis of the plurality oftarget images, and wherein a time duration of the moving image isdetermined on the basis of the detection time estimated by the timeestimator.
 9. The facial image display apparatus according to claim 1,wherein the path comprises a sequence of the detected areas containingfaces of the plurality of target images according to one of the one ormore identified persons.
 10. The facial image display apparatusaccording to claim 9, wherein the moving image zooms in on the detectedfacial areas of the plurality of target images according to thesequence.
 11. A facial image display method executed on a computercomprising: detecting areas containing faces in a plurality of targetimages each displaying one or more faces, wherein each target image is astill image; automatically identifying one or more persons correspondingto the areas containing faces and creating, using the areas containingfaces, a path through the plurality of target images connecting thedetected areas containing faces according to the one or more identifiedpersons; and creating a moving image in which a camera moves through theplurality of target images according to the path, wherein the movingimage contains the areas containing faces from the plurality of targetimages.
 12. A non-transitory computer readable medium having storedthereon a program comprising the steps of: detecting areas containingfaces in a plurality of target images each displaying one or more faces,wherein each target image is a still image; automatically identifyingone or more persons corresponding to the areas containing faces andcreating, using the areas containing faces, a path through the pluralityof target images connecting the detected areas containing facesaccording to the one or more identified persons; and creating a movingimage in which a camera moves through the plurality of target imagesaccording to the path, wherein the moving image contains the areascontaining faces from the plurality of target images.